At least some known computing systems include remote client(s) or browser(s) that support various mechanisms for receiving data from a server, wherein the data may include screen changes. U.S. Patent Application Publication 2012/0324358, published Dec. 20, 2012 and incorporated herein by reference in its entirety, describes in detail an exemplary protocol for remoting a graphical user interface from a virtual machine to a client web browser via HTTP GET requests. Each GET request form the client web browser is matched to a screen update, which comprises an image taken from a portion of the graphical user interface that has recently changed due to applications or the operating system drawing to the display frame buffer.
U.S. Patent Application Publication 2013/0050253, published Feb. 28, 2013 and incorporated herein by reference in its entirety, describes a technique for compositing image data comprising overlapping display update images on a display screen at a remote user terminal using a standard web browser. As described, areas of a graphical user interface (GUI) that is modified by applications or the operating system are identified and bounded by an update rectangle, which is read from a frame buffer at the server and converted into an update image for display at a browser at a corresponding offset on the user's display, thereby updating the user's display with the GUI update generated at the server. When a subsequent image is received by the client it may be composited with previously received image updates by overlaying the new image on the previously received image data. The browser may internally maintain each update image as a separate entity such as an image object until a new image is received that entirely covers the previous image or even the entire display area.
Display remoting can adversely impact user experience, which is sensitive to display quality, latency, and frame rate. Display remoting protocols attempt to balance these factors given the available system resources, such as compute and network bandwidth. For example, the display remoting server may be configured to compress or encode the GUI updates prior to transmitting such data to the remote client(s) to accommodate the available network bandwidth. For example, the server may set the bandwidth budget per frame or per unit of time to an available number of bytes. However, it may be difficult to predict the quality level that the server or encoder therein should use when compressing an image, e.g., using JPEG image compression, to produce an image that will closely match the available bandwidth without exceeding it, thereby maximizing the user experience within the constraint of the available network bandwidth.
One approach to produce an image that matches a required output size is to repeatedly and iteratively encode the same image while adjusting the quality settings until the desired quality is achieved. However, this approach requires a relatively high central processing unit (CPU) and therefore a relatively high latency overhead. Another approach may be for the server to attempt to guess the quality setting based on the result of previous frames or images. However, this approach may not work when there is a sudden change in the image content being sent to the client(s). In all of these approaches, an initial low quality version of the image needs to be refined to high quality when spare bandwidth becomes available. Moreover, being able to later improve the image quality for regions of the screen that are already displayed but have not changed can be inefficient, as it may require duplication of data already sent.